The complexity of software programs has increased dramatically over the past twenty years. During this time, tools created to test and verify the various functions and operations of developed software and software under development have also grown in number, size, and complexity to keep pace with software development. The tools used for conducting a test on logically or physically separate software components, or a complete software program with many components often form an environment together with other hardware, instrumentation, simulators, emulators, and other support elements to form a “test bed.”
However, a test bed developed to exercise various functionality and operations of a developed software program often includes more test software or programs and analysis programs that can be executed in a reasonable amount of time. Moreover, new test programs and analysis programs are often added on a regular basis to increase the number of test programs executable forming the test bed. With such a test bed having a number of test programs and analysis programs it is often difficult and burdensome to determine which test or suite of tests to execute and in what order to carry out the testing and verification required of a new or modified software component that has been developed or is under development or a new or modified software program that has been developed or is under development.
Conventional methodologies for sorting and scheduling execution of test programs to verify the functions and the operations of software components and software programs along with the compliance of the software components and software programs to requirements or specifications has been based largely on a hit or miss approach often derived from historical experience or data. As such, it is often difficult to predict the quality of a test case using test program software selected in this manner as well as difficult to predict for execution time along with other constraints imposed on the test case such as cost.